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L INTRODUCTION 

Display and control techniques for avionics and large process control systems are undergoing 
dramatic changes as a result of three major factors: (1) demonstrated performance of well-designed 

systems that include non- redundant airborne processors with a me an -time -to -failure of about 40,000 hours 
(ref 1.) (2) the design of even more complex systems by applying this kind of component technology to 

computer systems organized to be fault tolerant and gracefully degradable (ref 2); and (3) availability of 
flight qualifiable general purpose interactive graphical display/control devices with significant capability 
and flexibility. 

The object of this paper is to discuss (1) display control considerations associated with these new 
techniques, (2) general purpose displays in contrast with previous techniques, and (3) a prototype interactive 
display /command design presently implemented featuring a pushplate CRT overlay for command input. 

II. DISPLAY AND RETRIEVAL OF STORED INFORMATION 

General purpose interactive graphical display and control techniques allow more flexibility in the 
manner data may be displayed and responded to by the operator. With conventional techniques, specific 
devices are dedicated to each subsystem and the system designer must decide how best to locate the 
various displays and controls under varying criteria caused by varying mission requirements (data needed 
for landing and takeoff not necessarily needed for cruise and navigation) and varying system modes 
(checkout/ nominal flight/ emergency). 

This fragmented approach to presenting information is largely attributable to technology limitations 
and reluctance to implement new and perhaps marginally verified techniques. Dedicated devices take up 
panel space according to their functions and are located according to the relative importance, size, and 
expected amount of use. As a consequence, cockpits are filled with a proliferation of dedicated devices 
which may be used but once or twice during the course of a mission. This distribution of devices and 
associated information we call "horizontal" in structure. That is, the instantly available information lies 
spread out before the operator, access being limited largely by the user's familiarity with the panel 
format. 

Major criticisms of this format include: (1) limited information can be provided; (2) panel space is 
taken up equally by devices used 10% of the time as by those used 90%; (3) the user has to sort through a 
maze of switches and dials to retrieve much of his information; (4) flexibility of display format is minimal; 
(5) danger of actuating or interpreting wrong device at the wrong time is greater than if unused devices 
could be "stowed" in some sense. A major desirable feature is that information is immediately available 
at the flick of the eyes, particularly important during emergencies. 

Newer techniques offer an alternative by permitting the information to be stored in a more "vertical" 
structure, i.e., currently required information is the only data provided; the unneeded data is kept out of 
sight. This is done by time sharing a device capable of displaying any one of a multiplicity of display 
formats. These devices have been referred to as "generalized" displays and can help fulfill the objectives 
of minimizing weight, space, component proliferation (cockpit clutter), and maximizing reliability. 

In the effort to generalize displays and controls, great care must be taken to minimize the effort 
required to extract "stowed" information, particularly in emergency contingencies. These new systems 
may be less forgiving than the more fragmented approach because the "unneeded data" is stowed. In 
addition, certain information is to be dedicated in any case, and reasonable algorithms should be developed 
to determine which data fall in this class. Suppression of the unneeded data can be done in a variety of 
ways. Determining which technique to use is a critical problem. Primary factors in dedicating devices 
are frequency of use and required immediacy of response, particularly in crew safety and mission success 
contexts. Practical solutions appear to include a judicious mixture of "vertical" and "horizontal" information 
presentation. 

Our design activities in developing comm and- control techniques for complex systems have primarily 
concentrated on potential space-borne applications. The hardware/ software design objectives for this 
work include: 

A. Enable crew performance as system supervisor of the following kinds of tasks: 

1) Mission Sequence Initiation 

2) Specification of data, constraints, performance options 

3) Definition of system objectives 

4) Overview of automatic system sequencing, control, and status 

5) Reconfiguration of subsystem interrelations 

6) Performance of specific backup tasks in case of hardware/ software malfunction; 

B. Enable significant real time (or updated) changes of displayed information without requiring alteration 

of the basic software display file length and structure (this places requirements on the design of 

display generators which drive reformattable display devices); 
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C. Provide an interactive crew/computer interface scheme which will require a minimal amount of 
dedicated hardware, which is easy to learn to use, and provides rapid access to all information 
available in the system. ~ “~ —— 

A major objective is to devise an interface information flow sufficient to enable rapid, accurate 
crew appraisal of the system state, and to enable the operator to override automatic systems when his 
heuristic, nonliteral decision processes demand it. The system should be designed to force some minimum 
crew attention to the vehicle to ensure adequate operator takeover capability as well as effective flight 
control. 

For any airborne computer system in the foreseeable future, there will be strategy limitations. 
That is, we will not be able to predict all possible events to be encountered (except in the most finite of 
systems and environments) and we will not be able to optimize decision logic for all options opened to the 
crew. These limitations are derived largely from limited knowledge in planning an operational algorithm 
and limits in computer technology (speed and resolution). Hence, data regarding systems’ status and 
environment factors will have to be readily available at all times, and the data will have to be structured 
so as to maximize the ease of interpretability. This means integrating data such that the crew is assisted 
in perceiving key relationships among data, but without obscuring other relationships that may not have 
been foreseen "a priori”. 

A basic assumption is that the crew is in ultimate control i.e., the system will not take him where 
he does not want to go. The crew is the ’’flight controller” and the comm and- control system is his tool 
to permit efficient achievement of the mission/task objective(s). The question of how much automation is 
too much in a given instance can be largely a value judgment, depending on crew confidence in the 
instrumentation and general philosophy on the degree to which the system should be controlled manually. 
Hence, flexibility in utilizing automatic and manual modes in these difficult-to- resolve areas is mandatory. 
The automated functions should be implemented to appear to operate in a rational or reasonable manner. 
This and reliability are probably the two prime elements in fostering high user confidence in the system. 



1 Returnable to Main Index 
Can call Vehicle Data or special computation routines 


Figure 1 General Crew/ Computer Figure 2 Stored Display Formats 

Interface Structure 

HI. POSSIBLE COMMAND-CONTROL TECHNIQUES 
A. Data Presentation 

For spaceborne applications, we have found it useful to divide the information required by the operator 
into the following functional units: (1) major mission phase sequences, providing the overview and 

macro- command capability for mission phase progress, including comparison of alternative solutions 
provided by the integrated computational system as available; (2) vehicle state, a subset of which will 
normally be key parameters in cueing the operator to the status of mission sequences; (3) subsystem(s) 
status and reconfiguration data/ control. Some of the information implicit in the latter two categories 
may not be of sufficient priority to be kept up to date at all times. For this reason, a fourth group of 
information may be considered — special computation routines which will be available on call to the operator 
to provide vehicle and systems state information in special circumstances. 

Considering the general structure of the operator's information requirements and the nature of its 
storage (spread about in several subsystems and separate computation compartments), we have chosen 
an interface scheme as diagrammed in its general form in Figure 1. This is a tree-like structure with 
links across branches, enabling call of special data from the midst of (and return to) major mission 
sequences. The operator can recycle to the head of a sequence and to step ’’forward” through data display 
sequences. No piece of data is more than four keystrokes removed from a main line display. 

Other techniques which have been considered include a simple callable master index frame where 
all the data pages of interest would be callable by a coded indexing scheme (Fig. 2). Another technique is 
a two level identifier coupled with arbitrary numeric code for each option or data set (as used in the 
onboard Apollo programs) (ref 3 & 4). A third technique would be a hierarchical structure incapable of 
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indexing between the lower elements of the tree branches. Such a technique has been used in library 
retrieval schemes. 

B. Command Input 

Several alternative input techniques have been considered, including various keyboard arrangements. 
It was decided to implement a transparent display screen overlay enabling the operator to point at a 
desired option drawn on the display. This overlay design was motivated by the desire to pursue the 
concept of maximum crew/ display/ command integration to its reasonable limits. Other constraints to be 
satisfied include: 

1) Preclude the need for extra equipment to be handled by the operator (e.g,, light- or sona-pens); 

2) Don't reduce display image quality; 

3) Create keyboard "feel" to extent possible; 

4) Enable crew operation in pressurized garments. 

The overlay "keyboard' 1 consists of an 8 x 8 matrix of focussed light beams at one inch intervals 
sensed by photodetectors, all of which are mounted on the periphery of the display screen. Breaking a 
pair of the light beams by pointing at the display and pressing a pushplate about 1/16" toward the screen 
constitutes a ’’keystroke" input. 

C. General Description of the Prototype 


1. The prototype interactive control /display system has been fabricated to be used as a tool for 
evaluating alternative reformattable command/display techniques. Earlier reports have 
presented detailed descriptions of the hardware underlying this effort (ref 5 & 6). Briefly the 
basic components are (Fig. 3): 
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Figure 3 Display/ Command Prototype 
System Elements 



Figure 4 Prototype Display Sequence 


a. Display Output Processor (graphics generator), an 8K X 16 bit 2 ps instruction time, 
read only memory list file processor, the output of which defines the beam position and 
intensity on the CRT. Nested subroutining and indexing and hardware rotation permit 
direct means to dramatic changes in display formats with minimal coding and significant 
reduction in execution time. All display elements are created with straight line vectors. 
Display segments may be altered in size, intensity, by flashing on/off, and rotated. 

b. PDP-9 computer (18 bit, 16K, 1.2 us) part of whose memory is used by the DOP for 
refreshing the display; also stored here are the executive operating program, a simulated 
environment, and coding to update the displays. 

c. A 12" square Kratos CRT (P7 Phosphor) modified with a 3-bit intensity input is the 
display device. Display formats are 8" square. 

d. A transparent CRT overlay pushplate is the input device, described above. 

2. The programmed interactive control scheme in its general form is shown in Fig. 1. Specific 
displays and their interrelationships are diagrammed in Fig. 4. This approach has been taken 
to satisfy many of the considerations discussed earlier, and to explore possible useful techniques 
for data display and sequence control in the context of the space shuttle. A more detailed 
description of the programmed system's salient features follows. 











PRIME 


DAP 


CALL DATA 


D&C 


LAUNCH 


RENDZ 


ATTITUDE 

CONTROL 


ORBITAL 

NAV 





IRU 

ALIGN 


ORBIT 

CHANGE 

i 

ENTRY 


CRUISE/ 

LANDING 






1 

RENDZ 


■ 1 



OPTION 2 


1 



Figure 5 


An 

CONTL 


NAV 



RCS 


RADAR 


RECYCLE 

TO 

PRIME 


IRU 


ECS 



AIR 

DATA 


COMM 



CALL 

DATA 


RETURN 
TO LAST 
DISPLAY 


Figure 6 


The main index or "Prime 11 display (Fig. 5) provides a selection matrix of mission phase functional 
sequences, any one of which the operator may select with a single finger stroke. Options to review 
vehicle state or to call special computation subroutines, as well as return to the prime display, may be 
exercised at any time. Having entered a sequence, the subsequent displays provide all available options 
as the operator progresses through the mission phase. Nominal options are designated by underlining 
and flashing the square "key" associated with the option. 

If the "Call Data" option is selected, another index of options is provided (Fig. 6). This consists of 
an array of subsystems and functional units under which a library of data is stored in memory. These 
show fixed and dynamic data, representing vehicle state and/or subsystem(s) status. In all of these operations 
a miskey by the operator may be corrected with a single rekey stroke. 

The Prime or sequence display from which the "Call Data" matrix is called is scaled down and 
stored in the upper left of the CRT screen. The intent is to keep the operator informed regarding where 
in a sequence he is, rather than relying on his memory. This is particularly attractive in high stress 
situations or if the operator must direct his attention elsewhere having entered the Call Data block. 

All data are displayed in base 10 only and all formats include word length and decimal point location. 
More than one parameter is assigned to each display format. For those cases where crew alteration of 
the data is required, an up /down pointer cursor and numeric calligraphic keyboard have been implemented. 
The cursor may be moved one parameter at a time with a single keystroke. In those cases where a data 
category contains too many parameters to fit conveniently in one format, the cursor may be used to "turn 
the page" when the top or bottom of the current list is passed. 



Figure 7 


Figure 8 
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A calligraphic numeric keyboard (Fig. 7) has been implemented to enable alteration of data associated 
with the Call Data lists and other data alteration. As the data is keyed in, it appears in the buffer zone of 
this display. Not ‘until "ENTER" is keyed is the computer erasable memory location(s) altered or the 
display changed onHhe data page. Miskey may be corrected by hitting "CLEAR". 

Also available from the Call Datamatrixis the ability to reconfigure multiple redundant components 
of a simplified display subsystem. This display/ command format (Fig. 8) enables operator control of 
CRT/graphics generator connections as well as control of on/off status and parameter review with a 
minimum of pushplate keystroke activity. 

This overlay technique provides a simple means of providing a representation of on/off switches. 
An example reaction control configuration for a digital autopilot similar to that on the Apollo CSM is 
shown in Fig. 9. 



Figure 9 

As experience is gained with this system, those functions which could be profitably dedicated to 
their own input key are being identified. These include "Recycle to Prime" and "Call Data", functions 
useful at all displays. In an expanded operational context such functions as engine control (on/off) and 
other crew/mission safety items should be dedicated and probably redundantly hardwired around the central 
processor(s). 

Solid state keys mounted at the periphery of the CRT are another configuration under investigation. 
This would reduce the number of available options in any given display, but experience to date has not 
demonstrated the need for all those available with the pushplate overlay. The solid state keys have a 
superior subjective feel and eliminate the tendency to cover up the option being selected as it is keyed in. 
A mix of overlay and solid state keys and reformattable solid state keys are other configurations being 
investigated. 

Critique of Present Design 


The prototype interactive control scheme as fabricated offers advances over conventional designs 
including: 

1) Direct, obvious relation between the information displayed and choice of executable options. 

2) No shift of focus of attention between display of information and execution of choice. 

3) Fewer keystrokes to achieve retrieval of data. 

4) Data retrieval without using numeric codes as symbols for subsystems, data, sequences, etc. 

5) Performance of subsidiary tasks /displays without erasing present main sequence display from 
CRT screen. 

6) Inclusion of data associated with a large number of conventional dedicated display devices. 

The general structure of the scheme is sufficiently intuitive that learning problems to date have 
been minimized. The design can be augmented by including teaching aids as required, or integrating 
more traditional check-list type material into the displays as desired. Further perfection of the mechanical 
reliability of the overlay and reduction of viewing paralax caused by the distance between the CRT screen 
and the pushplate should be pursued, however. 

Some of the potential difficulties with this kind of design include: 

1) Designer's inability to anticipate all data parameters which might have to be immediately 
available to an operator in an unforeseen emergency may result in undesirably long sequences 
to retrieve a piece of data or establish alternative active configuration(s). This can be alleviated 
perhaps by structuring the data network in a more horizontal format — use of more display 
screens. 
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2) Not all users may find a particular sequence to retrieve a data point stored at a lower level 
very obvious. Probably care should be taken to design the system to enable retrieval of 
particular bits of information by more than one route through the structure. 

3) In unanticipated situations where many parameters are having to be reviewed virtually 
simultaneously, forcing the operator to switch back and forth among displays may prove 
unacceptable. This can be alleviated by suitable display device redundancy, which will probably 
be forced by hardware reliability considerations, in many cases. 

4) The possibility exists that even though the time and effort to reach data at the nth level may 
be small compared with more traditional techniques, it may be more irksome to the user 
because of perceived remoteness of the data. It may prove that having to look up a code in a 
book and key it in is more satisfactory subjectively than being led down a 3 or 4 or more 
level path. 

V. OTHER POSSIBLE APPLICATIONS 
PROCESS CONTROL 


Process control facilities are usually highly automated systems which perform quite precise and 
rather narrowly- defined tasks. A human monitor/operator is appended to deal with those contingencies 
which are so diverse and occassional that it is unreasonable to automate the system to deal with them 
unaided (ref 7). 


The human typically scans a handful of critical parameters to verify acceptable performance The 
operator is required to perform at a reasonably complex interface and deal with data immediately at 
hand rather than rely upon repeated practice sessions dealing with the same contingency. He must perform 
under stress and be able to control the timing of his actions. To deal with boredom, wandering attention 
and time delays in familiarizing himself with the current state of an aberrant system, non- static information 
and new forms of communication need development. This kind of situation in which operators must act 
quicKiy on very occassional bases requires communication schemes which will refresh the operator's 
understanding of his available options and help him judiciously filter out that information which is not 
immediately relevant. 

COMPUTER AIDED INSTRUCTION 


The requirement for immediacy of student- system communication is critical and obvious. The 
goal of minimizing the amount of relatively complex keyboard activity to be learned just to start to use 
the system is clear. The ability to present all and only those options pertinent to a particular question 
or subject is highly desirable. The ability to jump readily to another learning sequence as ideas germinate 
i s °* P°t entia l attraction. The generalized display/ control concept described here goes a long way to 
fulfilling these desirable characteristics in a self-paced instruction context, although forced paced 
instruction/ testing may be desirable under some circumstances.. 


MEDICAL PATIENT HISTORY/ SYMPTOM SELF- DESCRIPTION 


To the extent that the patient can describe his symptoms and past medical history without direct 
contact with a doctor and without undue time spent and incidence of error, we will have reduced one 
significant part of the doctor 's burden. This application is similar to the computer aided learning situation. 
The patient is asked a series of basic questions about his health status, the answers to which lead to 
with°the^ patient nqUeS tions* The doctor may build upon this basic framework with personal communication 


touch P° int . overlay for the patient's information input eliminates the need to know how to type or 
^ , + d . evices - J he storage of the data eliminates problems associated with 

phering of handwriting. Such a system has in fact been tried in areal medical environment with 

promising preliminary results (ref 8). 1 

Provision for definition of unfamiliar terms, forcing the user to acknowledge understanding of the 
qu stion and/or key terms are elements which should be included in this and the student learning context. 

VEHICLE DISPATCH 


and distrihiHno °P eratl ° ns typically consist of one or more dispatchers responsible for routing 

^distributing vehicles to need points as a function of constantly-changing real-time demand. The ability 

th ? tot .^ °P eratl0n 311(3 zero m on the status of particular vehicles and neighborhood vehicle 
distribution densities is required. In order to help dispatchers associated with police, taxi, trucking 
nf kPvtl^ bU anCe facilities to communicate effectively with a useful data retrieval system, elimination 
hoH^nn+pi H S a +t PPearS \°^ e a Particularly attractive characteristic. These operations appear to rely on 

scale disdavs 7 V*® extent the y do - computer overlay interface compatibility with large 

scale displays will be important elements in a useful system design. 

SUMMARY 


functions ^d stratiX OUrselves directly to the problems of manual override of automatic 

rather cor^lete^nw' /^ 0 ^ consequences in this new context. A full-scale simulation including 
wi h^ r aiire P stV™H P ^ entatl ° n • representative subsystems operating in time-critical mission phases 
the different* h«+ d fu requlred for a thorough evaluation of such factors. Further effort to quantify 
innUrfhlm / b f tween u the Present and alternative designs is planned. Also, investigations of the 
applicability of such a pushplate interactive control scheme toother fields of application are being pursued. 
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